1. Field of the Invention
The present invention relates to a data presentation system for such data as visual image data and audio sound data, and more particularly, to a data presentation system suitable for the so called virtual reality data presentation.
2. Description of the Background Art
Conventionally, there has been an image display for displaying the computer graphic images on a display screen of a work station at which the displayed images can be manipulated, and such an image display has been utilized in various designing purposes.
More recently, there has been a development of a virtual reality image display device having an input unit in a form of a data glove or a data suit which measures the body motion of an operator directly and inputs the measured data as input data for an input coupled image, such that the operator is allowed to make an active access to a virtual space formed within the displayed images in real time.
In such a virtual reality image display device, the 3D (three-dimensional) images are displayed, and the objects shown in the displayed images can be lifted up, twisted, or pulled within the virtual space, in response to the corresponding operations of the input unit made by the operator and detected by the input unit, by reflecting the operations of the input unit made by the operator in the input coupled image in the displayed images.
In a conventionally available virtual reality image display device, the correspondence between the displayed image and the operations of the input unit by the operator have been made as follows.
Namely, the position of the input coupled image is determined in terms of a displacement from an origin of the input unit. In this case, the operation of the input unit always coincides with the motion of the input coupled image in the virtual space, so that the motion in the displayed image may appear unnatural or unreal in some cases.
For example, consider a case of using the data glove as the input unit and a hand image as the input coupled image, where the position and configuration of the data glove operated by the operator correspond to the position and the configuration of the hand image in the displayed image. In this case, when the operator carries out the operation to pick up an object in the virtual space and displaces it to the other location within the virtual space, the operator's hand operating the data glove is freely movable in the real space, so that the object in the virtual space can be moved in response to the unintentional slight trembling of the operator's hand. However, such a motion of the object in the virtual space can appear unnatural in view of the real space in which the object has some weight or the object has some support so that the object cannot be moved so easily by such a slight trembling of the hand.
In this regard, in order to realize the more realistic presentation of the virtual space, an additional control mechanism for limiting the operation of the input unit by the operator may be provided such that this control mechanism is activated to increase the reality in the virtual space. (See, H. Iwata, "Artificial Reality with Force-feedback: Development of Desktop Virtual Space with Compact Master Manipulator", Computer graphics, Vol. 24, No. 4, August 1990.)
For example, by providing a feedback for controlling the input unit to the input unit according to the positions of the hand image and the object in the displayed image, the operator's operation of the data glove can be appropriately limited when the object is heavy, such that the sense of a reality in the operation of the input unit can be realized, and consequently the displayed image can appear more natural. In addition, because of the presence of the feedback to the input unit, it also becomes possible for the operator to sense the situation within the virtual space.
However, such an additional control mechanism requires a highly specialized equipment to be attached to the input unit, which also makes the input unit as a whole to be quite large, and its applicability and operational range can be quite limited, so that this scheme is not very practical.
There has also been a problem concerning a relationship among the objects in the displayed virtual space. Namely, in a conventional virtual reality image display device, because the objects in the virtual space can be moved by the operator through the input unit, the interference checking has been carried out in order to avoid the unnatural display of the objects overlapped with each other. For example, when a box located on top of a table is to be moved to another table in the virtual space, the box should not appear to be overlapping with another table after the box has been moved, and should properly appear to be located on top of another table.
Although this interference checking is capable of avoiding the unnatural display very strictly, it requires a large amount of calculations for determining the display positions of the objects, so that it is quite time consuming. In particular, when the virtual space contains many objects, the amount of calculations required by the interference checking can be so enormous that the real time processing of the display in response to the operation of the input unit by the operator becomes practically impossible.
Now, consider a case in which the operator is to move through the virtual space representing 3D model of the interior of a building or a city. In such a case, it is often difficult for the operator to comprehend where he is located presently and which way he is oriented presently, so that there is a need to provide such a position information to the operator. Here, the position information useful for the operator includes the operator's viewpoint indicating the present position in the displayed virtual space, the operator's reference point indicating the object presently attracting the operator's attention in the displayed virtual space, and the operator's field of view indicating the range that the operator can presently see in the displayed virtual space.
Conventionally, such a position information has been presented either in terms of numerical data for the viewpoint, the reference point, and the field of view, displayed at a part of the display, or in a form of a two-dimensional overview of the entire virtual space looked from a particular direction on which the viewpoint is indicated by a dot and which is displayed at a part of the display.
However, these conventional manners of presenting the position information have been associated with the drawbacks in that the numerical data are very difficult to comprehend immediately, and that the overview can only provide the very rough information, so that the more informative information such as what is located in a vicinity of the operator's present position cannot be obtained from the overview.
Now, in a conventional computer graphics technique, the shape of the object and the material of the object are usually defined separately. For example, when the 3D shape data for a pot are provided, the material property of this pot can be quite different depending on a texture with which its surface is presented.
The simplest method for presenting the material property of an object is to change the color and the amount of light reflection of its surface. For example, when the surface makes a mirror like reflection of lights, the object can appear metallic. On the other hand, when the surface makes a uniform scattering of the lights, the object appear can plastic. However, this simplest manner can only present a uniform surface quality.
Another conventionally known method for presenting the material property of an object is a so called texture mapping, in which the patterns indicative of the texture is attached on a surface of the object. For example, when the characteristic pattern of the grain of the wood is attached to an object, the object can appear to be made of wood, whereas when the characteristic pattern of marble is attached to an object, the object can appear to be made of marble. It is also possible to attach a picture such as a label. However, in this texture mapping method, only a flat pattern is attached to a flat surface, so that the material property of concavity or convexity cannot be presented.
In order to make up for this shortcoming of the texture mapping method, there is also a method called bump mapping, in which a divergence of normal vectors is attached to a surface of an object. In this bump mapping method, it is therefore possible to provide an uneven appearance of the surface, which can be put in a pattern of a rock face, a brass work, or a water ring. Even in such a case, there is no need to complicate the original object shape data in this bump mapping method.
Yet, these texture mapping method and the bump mapping method are exclusively directed toward the presentation of the visual material property alone, despite of the fact that the object in the real space has many other characteristic material properties. For example, a wooden pot not only has a pattern of the grain of the wood on its surface, but it also makes a dull sound when it is hit, and it has rather rough feel when it is touched. The texture mapping method and the bump mapping method described above totally ignore these other material properties of the objects.
On the other hand, in the recent virtual reality data presentation, the virtually realistic situation is realized by utilizing various material property presentation techniques such as 3D image display, computer graphics, 3D sound space production, and artificial touch sense presentation.
By using this technique of the virtual reality, it is possible to provide an opportunity for anyone to experience an event that is rather difficult to experience in real space. Also, this virtual reality technique can be utilized in as a simulator for the various trainings such as a flight training, such that the training can be carried out in a realistic situation.
In a conventional virtual reality data presentation system, various presentation parameters for determining the manner of data presentation such as the viewpoint for the image display, the repulsive force to be given back, the loudness of the sound to be outputted, etc. are designed in view of a prescribed standard user's attributed values such as height, strength, etc., so that a user having the attributed values which are greatly different from those of the standard user may not be able to receive the presented data to be natural because of the setting of these presentation parameters which seems unbalanced or distorted to such a user.
For example, when the viewpoint of the image display is designed in view of an adult male user, a child user who has much shorter arm length compared with the adjust male would have a difficulty in reaching to an object which is designed to be placed in the displayed virtual space at a position reachable by a standard adult male user. Also, in a case there is a repulsive force to be given back from the data presentation system to the user, the repulsive force designed in view of an adult male user can be dangerously strong for a child user.
Similarly, in the virtual reality data presentation system using the 3D image display or the 3D sound space production, the presented images and sounds may not necessarily have the truly 3D quality for some users who have attributed values greatly different from those of a standard user in view of whom the system had been designed.
Also, in a case of utilizing the virtual reality data presentation system for evaluating the proto-type product, it is preferable to make an evaluation from a standpoint of the actual user of the product. Thus, when the product is a toy to be used by a child, it is preferable to make an evaluation from a standpoint of a child user who has much different attributed values compared with an adult who has developed the product. Similarly, when the product is a kitchen system, it may not be sufficient to make an evaluation from a standpoint of an adult female in general, and it is rather preferable to make an evaluation from a standpoint of a younger adult female, a standpoint of an older adult female, etc. who have largely different physical attributed values as well as greatly different preferences.
However, the conventional virtual reality data presentation system has been designed in view of a specific standard user, and it has been impossible to adjust the settings of the presentation parameters according to the attributed values of the individual user or to experience the different standpoints of various different users.
Now, conventionally, the training of an operator of a monitoring system for a nuclear plant or a steel plant, has been achieved by using a replica of the actual monitoring system, such that the training for various situations including those of the serious troubles can be carried out, while the actual monitoring system remains in operation.
However, such a replica monitoring system for training purpose is made in the actual size of the actual monitoring system, which is quite large. In addition, a number of different replica monitoring systems must be provided for a number of different monitoring targets, so that this scheme of training can be quite costly. In addition, at a time of updating the replica monitoring system in accordance with the updating of the actual monitoring system, the physical construction works must be carried out, so that this scheme is both time consuming as well as ineconomical.
In view of such problems encountered in the conventional training scheme suing replica, the use of the virtual reality data presentation system as a simulator for such a training is increasing. In this application of the virtual reality data presentation system, the simulated monitoring system can be updated easily as it is realized as 3D model on a computer, so that it is much less time consuming as well as less costly.
In such a virtual reality data presentation system, hen the straightforward 3D model presentation alone is not realistic enough, the reality of the presented data can be increased by the method called texture mapping in which the texture expressing the material property of the objects in the 3D model is attached to the objects in the 3D model, as described above.
Now, in this texture mapping method, the texture to be attached to the object is usually obtained as images taken through an image scanner. For this reason, the attached image itself remains unchanged regardless of whether it is looked at from far away or nearby. However, in the real space, the each small pat-tern in the image is not clearly visible and appears to be absorbed into the image as a whole from far away, while it becomes clearly and distinctively visible from nearby, and such a perspective cannot be obtained by the texture mapping.
In order to improve in this aspect, there is a proposition for switching one texture image with another texture image in which the pattern is shown more clearly whenever the distance from the viewpoint becomes less than a prescribed value. However, in this modified method, there is a drawback in that the switching of the images must be made so abruptly that the large changes of the details of the appearance before and after the switching can make it appear rather unnatural. In order to achieve the smooth changes of the appearance, a large number of images must be provided and the switching operation must be repeated.
On the other hand, the monitoring screen on the monitoring system shows various data given in terms of letters, figures, and graphs from which the operator makes judgements, these data are constantly changing in time in the real space, but the image attached by the texture mapping is unchangeable, so that the various data necessary for making judgements cannot be provided to the operator in the simulated monitoring system using the texture mapping method.
In order to overcome this drawback, there is a proposition for using the real computer output. However, in such a scheme, the data provided by the real computer output are practically invisible when the viewpoint is located far away, so that the time and the system function for maintaining the display of the real computer output is wasteful in such a case.
Furthermore, in a conventional virtual reality data presentation system, the presented data are largely biased toward the visual image data. However, in the actual monitoring system, the audio sound data such as the vibration sounds of the motors and the sounds of fluid flows through the pipes can provide the variable information, so that there are cases in which the actual sounds collected by the microphones located at the monitoring target cite are outputted at the monitoring system through speakers.
However, in a case the number of monitoring targets increases, if all the sounds collected from all the monitoring target are outputted continually, the slight abnormal sound indicative of the malfunction can be inaudible because of all the sounds outputted concurrently.
Now, the designing the various office rooms and monitoring rooms has conventionally been made on a basis of a sketches from various viewpoints prepared by a designer. However, it is practically impossible to prepare the sketches from arbitrary viewpoint, accounting for every possible settings of the variable parameters, so that the designed arrangement can be evaluated only from a limited number of viewpoints.
In order to resolve this problem of the conventional designing procedure, it is possible to utilize the computer graphics technique for the designing purpose.
In this case, the displayed 3D model representing the designed arrangement can be prepared by using the CAD system and displayed in a form of a perspective projection from arbitrary viewpoint, so that the designed arrangement can be evaluated more thoroughly.
However, even when the replacement of the designer's sketches with the computer graphics is made, it is still impossible to check the reflections due to the ceiling lights on the monitor screens, which is very serious practical concern.
Moreover, in the conventional computer graphics, even when the defect of the arrangement is found on the displayed 3D model, it has been impossible to make the necessary corrections on the displayed 3D model, because the computer graphics is basically the display tool which utilizes the 3D model data solely for the purpose of displaying.
On the other hand, the human comprehension of the 3D arrangement of the 3D objects is largely helped by the presence of the shadows on the floor. In the conventional computer graphics, there are several shadow calculation algorithms including the ray tracing algorithm based on the mirror reflection model, and the radio city algorithm based on the total scattering model. The former is known to be suitable for the reflection at the metallic surface, while the latter is known to be suitable for calculating the soft shadow images.
However, these conventionally known shadow calculation algorithms have the serious drawback of the enormous time required for their calculations.
There has also been a proposition for the high speed shadow calculation algorithm called fake shadow algorithm. however, this algorithm is capable of producing only very clear shadow images which may not necessarily be very realistic.
Now, in the monitoring room or the office room which is equipped with many ceiling fluorescent lights, the actual shadows on the floor appear rather dim, with uncertain boundary. Thus, the shadows calculated by the fake shadow algorithm appear too distinct to be natural.